High - level Vision as Statistical Inference

Human vision is remarkably versatile and reliable, despite the fact that retinal image information is noisy, ambiguous, and confounds the properties of objects that are useful. By treating vision as a problem of statistical inference, three classes of constraints can be identified: the visual task, prior knowledge of scene structure independent of the image, and the relationship between image structure and task requirements. By considering the visual system as an organ for statistical inference, we can test whether and how it uses these constraints. This strategy is illustrated for two high-level visual functions: depth-from-cast-shadows and viewpoint compensation in 3-D object recognition. An object’s relative depth can be determined from its cast shadow, even when local image information doesn’t uniquely specify shadow edges, and global information doesn’t determine where the light source is. What information enables a unique estimate of depth from shadows? This chapter shows how the visual task, prior assumptions on light movement and material properties, and local image cues constrain the perception of depth from shadows. A 3D object can be recognized from views never seen before, despite the fact that depth information about shape is lost due to projection on to the retina. How does human recognition compensate for variations in viewpoint? By designing a simple recognition task for which optimal statistical decisions are computable, human performance can be normalized with respect to the information in the task, leaving remaining differences diagnostic of brain mechanisms.